Flaw inspection apparatus and method

ABSTRACT

A flaw inspection apparatus according to the present invention includes a deep learning unit to which an image obtained by photographing a surface of an inspection object is input and in which, on the basis of the input image, the deep learning unit judges absence or presence of a flaw on a surface of the inspection object and specifies a site judged as being the flaw; a dimension measuring unit that measures a dimension of the flaw on the basis of the image of the site specified by the deep learning unit; and a flaw classifying unit that classifies the flaw on the basis of the dimension of the flaw measured by the dimension measuring unit.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based on Japanese Patent Application No.2019-024369, the contents of which are incorporated herein by reference.

TECHNICAL FIELD

The present disclosure relates to a flaw inspection apparatus and a flawinspection method.

BACKGROUND ART

An automatic inspection apparatus that inspects an inspection object byusing deep learning in a neural network is known (for example, seeJapanese Unexamined Patent Application, Publication No. 2003-76991).

SUMMARY OF INVENTION

According to an aspect of the present disclosure, there is provided aflaw inspection apparatus including: a deep learning unit to which animage obtained by photographing a surface of an inspection object isinput, in which, on the basis of the input image, the deep learning unitjudges absence or presence of a flaw on a surface of the inspectionobject and specifies a site judged as being the flaw; a dimensionmeasuring unit that measures a dimension of the flaw on the basis of theimage of the site specified by the deep learning unit; and a flawclassifying unit that classifies the flaw on the basis of the dimensionof the flaw measured by the dimension measuring unit.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram indicating a flaw inspection apparatusaccording to one embodiment of the present disclosure.

FIG. 2 is a diagram illustrating one example of an image obtained by acamera in the flaw inspection apparatus illustrated in FIG. 1.

FIG. 3 is a diagram illustrating rectangular regions that include sitesjudged as being flaws and that are obtained by inputting the imageillustrated in FIG. 2 into a deep learning unit.

FIG. 4 illustrates an image example in which the probability of therectangular region A illustrated in FIG. 3 having a flaw is expressed ingrayscale.

FIG. 5 illustrates an image example in which the probability of therectangular region B illustrated in FIG. 3 having a flaw is expressed ingrayscale.

FIG. 6 illustrates an image example of a binary image obtained bybinarizing the image illustrated in FIG. 4.

FIG. 7 illustrates an image example of a binary image obtained bybinarizing the image illustrated in FIG. 5.

FIG. 8 is a flowchart indicating a flaw inspection method that uses theflaw inspection apparatus illustrated in FIG. 1.

FIG. 9 is a block diagram indicating a modification of the flawinspection apparatus illustrated in FIG. 1.

DESCRIPTION OF EMBODIMENTS

A flaw inspection apparatus 1 and a flaw inspection method according toone embodiment of the present disclosure will now be described withreference to the drawings.

As illustrated in FIG. 1, the flaw inspection apparatus 1 according tothis embodiment is equipped with a camera 2 that photographs aninspection object O, and an image processing device 3 that processes animage (two-dimensional image) P1 obtained by the camera 2.

The image processing device 3 is equipped with a deep learning unit 31to which the image P1 obtained by the camera 2 is input; a flawdimension measuring unit (dimension measuring unit) 32 that calculates adimension of a flaw from the image output from the deep learning unit31; a flaw classifying unit 33 that classifies the flaw on the basis ofwhether or not the calculated dimension of the flaw exceeds apredetermined threshold; and a storage unit 34 that stores the flaw andthe dimension in association with each other. The deep learning unit 31,the flaw dimension measuring unit 32, and the flaw classifying unit 33are each constituted by a processor, and the storage unit 34 isconstituted by a memory.

In the deep learning unit 31, images P1 of a large number of inspectionobjects O obtained in advance and the information about absence orpresence of flaws in the images P1 are input to enable deep learning andconfigure a learning model.

Since the information used to configure the learning model only needs tobe information about whether there are flaws or not, there is no need tomeasure the dimensions of the flaws, and, thus, the learning task issimple.

When an image P1 obtained by the camera 2 is input to the learning modelin the deep learning unit 31, absence or presence of flaws on thesurface of the inspection object O in the image P1 is judged. FIG. 2illustrates an image P1 obtained by the camera 2, and FIG. 3 illustratesthe image P1 obtained by the camera 2 in which information aboutpositions of rectangular regions A and B containing sites judged asbeing flaws is specified by the deep learning unit 31.

The deep learning unit 31 then outputs information about the probabilityof there being a flaw for each of the pixels in the rectangular regionsA and B.

As illustrated in FIGS. 4 and 5, images P2 and P3 in which portions withhigh probabilities of being flaws are indicated in lighter shades andthose with low probabilities of being flaws are indicated in darkershades are generated in the flaw dimension measuring unit 32 on thebasis of the information output from the deep learning unit 31.Furthermore, as illustrated in FIGS. 6 and 7, the generated images P2and P3 are binarized by a predetermined threshold to generate binaryimages P4 and P5.

For each of white pixel regions in the generated binary images P4 andP5, the flaw dimension measuring unit 32 calculates at least one of thelength in the longitudinal direction, the length in a directionorthogonal to the longitudinal direction, the perimeter, and the area.

The flaw classifying unit 33 compares the threshold and the dimensioncalculated in the flaw dimension measuring unit 32. The threshold is avalue for judging, for example, whether repair is possible or whethershipment is possible, and can be set as desired.

In other words, flaws that have lengths, perimeters, or areas that areless than a predetermined threshold can be classified as repairable oracceptable for shipment, and flaws that have lengths, perimeters, orareas that are equal to or greater than the threshold can be classifiedas unrepairable or unacceptable for shipment.

The storage unit 34 stores the classified flaws in association with thedimensions measured by the flaw dimension measuring unit 32.

A flaw inspection method that uses the flaw inspection apparatus 1 ofthe present embodiment having the above-described features will now bedescribed.

As illustrated in FIG. 8, the flaw inspection method of this embodimentinvolves photographing an inspection object O with the camera 2 toobtain an image P1 (step S1), and inputting the obtained image P1 to thedeep learning unit 31. In the deep learning unit 31, absence or presenceof flaws on the surface of the inspection object O is judged (step S2),and the sites judged as flaws are specified (step S3).

When there are no sites judged as being flaws in step S2, the process isended. When there are sites judged as being flaws, information regardingthe probability of a flaw being present is output to the flaw dimensionmeasuring unit 32 for each of the rectangular regions A and B containingthese sites, and, for each flaw, at least one of the length, perimeter,and area is calculated in the flaw dimension measuring unit 32 (stepS4).

Next, whether the calculated dimension is equal to or greater than athreshold is judged in the flaw classifying unit 33 (step S5), and theflaws are classified into two categories, X and Y, respectivelycorresponding to flaws that have a dimension less than the threshold andflaws that have a dimension equal to or greater than the threshold(steps S6 and S7). The classified flaws are stored in the storage unit34 in association with the dimensions of the flaws measured in the flawdimension measuring unit 32 (step S8).

When not all of the rectangular regions A and B are classified, thesteps from step S4 are repeated for the next rectangular regions A and B(step S9).

According to the flaw inspection apparatus 1 and the flaw inspectionmethod of this embodiment, absence or presence of flaws is judged bydeep learning from an image P1 obtained by the camera 2. Thus, absenceor presence of flaws can be judged and the sites thereof can bespecified without clearly defining what flaws are. In other words,according to deep learning, whether the concerned object is a flaw orattached matter such as dust can be easily learned, and the absence orpresence of flaws in the input image P1 can be easily judged.

The sites judged as being flaws are subjected to image processing in theflaw dimension measuring unit 32 to measure at least one of the length,perimeter, and area for each flaw. Thus, the flaws can be easilyclassified in the flaw classifying unit 33.

In this case, a site judged as being a flaw in deep learning is measuredto determine the dimension of the flaw by image processing. Thus, thereis an advantage in that there is no need to use dimensions of flaws inthe learning stage of deep learning, and thus learning can be completedeasily and in a short time. Another advantage is that when the measureddimension is less than a threshold, and when, for example, the flaw isclassified as acceptable for shipment, the flaw and the dimension inassociation with each other are stored in the storage unit 34, and thusthe traceability after shipment can be improved. Yet another advantageis that the shipment standard can be adjusted by changing the thresholdof the dimension of the flaw without having to perform the learningagain.

In this embodiment, in the flaw dimension measuring unit 32, at leastone of the length, perimeter, and area of a flaw is calculated, and thecalculated dimension is compared with a threshold to classify the flawinto two categories, X and Y. Alternatively, all of the length,perimeter, and area of a flaw may be calculated, and the flaw may beclassified into two categories, X and Y, on the basis of whether any oneof these values is equal to or greater than the corresponding threshold.Depending on the types of dimensions that exceed the thresholds, theflaw may be classified into three or more categories.

In this embodiment, images P2 and P3 generated from the informationindicating the probability of constituting a flaw output from the deeplearning unit 31 is binarized, and the dimensions are measured fromthese binary images; alternatively, the image P1 obtained by the camera2 may be directly subjected to image processing so as to extract theedge of a flaw and calculate at least one of the length, perimeter, andarea of the flaw by using the extracted edge.

In this embodiment, the camera 2 captures a two-dimensional image P1;alternatively, the camera 2 may capture a two-dimensional image and athree-dimensional image. As illustrated in FIG. 9, the camera 2 may beequipped with both a two-dimensional camera 21 and a three-dimensionalcamera 22 so as to capture a two-dimensional image and athree-dimensional image by switching between these cameras.Alternatively, two two-dimensional images with different parallaxes maybe obtained so as to form a three-dimensional image from the twotwo-dimensional images.

In such a case, as illustrated in FIG. 9, in the deep learning unit 31,the two-dimensional images may be used to judge absence or presence of aflaw and to specify the site of the flaw, and, in the flaw dimensionmeasuring unit 32, the three-dimensional image may be used to measurethe depth of the flaw.

In this manner, the depth can be used as the standard for classifyingthe flaw. In the flaw dimension measuring unit 32, both atwo-dimensional image and a three-dimensional image may be used to useat least one of the length, perimeter, area, and depth of the flaw toclassify the flaw.

In this embodiment, a moving mechanism that moves the three-dimensionalcamera 22 may be provided. In this manner, the three-dimensional camera22 can be moved by the moving mechanism on the basis of the positioninformation of the flaw obtained from the two-dimensional image, andthus a three-dimensional camera 22 having a narrower field of view (forexample, a camera with a higher resolution but with a narrower field ofview) than the two-dimensional camera 21 can be used.

REFERENCE SIGNS LIST

-   1 flaw inspection apparatus-   31 deep learning unit-   32 flaw dimension measuring unit (dimension measuring unit)-   33 flaw classifying unit-   34 storage unit-   O inspection object-   P1 image (two-dimensional image)

1. A flaw inspection apparatus comprising: a deep learning unit to whichan image obtained by photographing a surface of an inspection object isinput, in which, on the basis of the input image, the deep learning unitjudges absence or presence of a flaw on a surface of the inspectionobject and specifies a site judged as being the flaw; a dimensionmeasuring unit that measures a dimension of the flaw on the basis of theimage of the site specified by the deep learning unit; and a flawclassifying unit that classifies the flaw on the basis of the dimensionof the flaw measured by the dimension measuring unit.
 2. The flawinspection apparatus according to claim 1, wherein the dimensionmeasuring unit measures at least one of a length and an area of the flawin a binary image obtained by binarizing pixel values used in judgingabsence or presence of the flaw in the deep learning unit.
 3. The flawinspection apparatus according to claim 1, wherein the dimensionmeasuring unit extracts an edge of the flaw in the image and measures atleast one of a length and an area of the flaw on the basis of theextracted edge.
 4. The flaw inspection apparatus according to claim 1,wherein the input image includes a two-dimensional image and athree-dimensional image; on the basis of the two-dimensional image, thedeep learning unit judges absence or presence of the flaw on the surfaceof the inspection object and specifies the site judged as being theflaw, and on the basis of the three-dimensional image, the dimensionmeasuring unit measures a depth of the flaw at the site specified by thedeep learning unit.
 5. The flaw inspection apparatus according to claim1, further comprising a storage unit that stores the flaw classified bythe flaw classifying unit in association with the dimension of the flawmeasured by the dimension measuring unit.
 6. A flaw inspection methodcomprising: inputting an image obtained by photographing a surface of aninspection object; on the basis of the input image, judging absence orpresence of a flaw on the surface of the inspection object andspecifying a site judged as the flaw; measuring a dimension of the flawon the basis of the image of the specified site; and classifying theflaw on the basis of the measured dimension of the flaw.